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Summary of Discussions with Mr. Chris Vo 



1 . In a telephonic conversation on June 17, 2004, Mr. Chris Vo informed the 
Examiner that the final Office Action was sent to the wrong address and requested 
additional time for preparing a response to the Action. Mr. Vo stated that a power of 
attorney and change of address had been submitted on August 27, 2003, and that the 
Office Action was not sent to the address submitted. The Examiner stated that no 
power of attorney or change of address had been entered in the system, but he would 
review the case. After reviewing the case, the Examiner informed Mr. Vo on June 22, 
2004, that the change of attorney and change of address was received but not entered. 
He also told Mr. Vo that they would be entered, and that another final office action 
would be mailed, with a new shortened statutory period set. 

Note Regarding The Current Office Action 

2. This Office Action replaces the previous Office Action, paper number 18, the 
latter having been sent to the wrong address. 



3. The amendment filed August 27, 2003, has been entered and made of record. 

Applicant's amendment addresses the issues raised in the rejection under 35 
U.S.C. § 1 12, second paragraph. However, see the new rejection on the same ground 
due to the amendment to claim 57. 



Response to Applicants' Amendment and Arguments 
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Applicant's arguments have been fully considered, but they are not persuasive 
for at least the following reasons. 

On page 18, next to last paragraph, Applicant argues that Fisher and Reinhardt 
do not pertain to analogous environments. The Examiner disagrees. They are at least 
analogous because Reinhardt relates generally to allowing a model to be matched up 
with an underlaid photograph to give the impression that the model is part of the 
photograph (column 2, lines 22-25), which is similar to Fisher's invention. Additionally, 
both patents are classified in the same class (class 345) of the classification schedule of 
the U.S. Patent and Trademark Office. Patents within the same class of the 
classification schedule correspond to related subject matter. 

In the paragraph bridging pages 19-20, Applicants argue that Reinhardt does not 
teach step f) of claim 13. Applicants state that column 7, lines 62-65 (cited by the 
Examiner) pertain to steps c) and d) of claim 13. The Examiner responds by pointing 
out that further reading from column 7 line 66 through column 8 line 10 shows 
Reinhardts' teaching of step f). Note that Reinhardt discloses calculating the 
geometric coordinates of each primitive (the objects) and projecting through the virtual 
camera to yield an updated projected wireframe representation overlaid on the image. 
This updating essentially amounts to a transforming. 

On page 25, first full paragraph, with regard to claim 26, Applicant argues that 
neither Fisher nor Reinhardt teach step f) of claim 13. The Examiner responds by 
pointing out that claim 26 does not have a "transforming" step which corresponds to 
step f) of claim 13. 
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On page 25, first full paragraph, Applicant states that claim 26 has been 
amended to receive camera parameters for the digital image of the space from the 
photogrammetric modeling engine, and an input to receive one or more product 
representation for selected products not of the space to be inserted into the space. The 
Examiner considers this to still read on the combination of Fisher and Reinhardt. As 
discussed in the previous office action, Reinhardt provides for the camera parameters. 
Note also that the intent of Fishery invention is to allow selection of products not of the 
space and to insert them in the space. 

On page 25, last full paragraph, with regard to claim 28, Applicant refers to step 
f) of claim 13. However, the Examiner notes that claim 28 does not have a step which 
corresponds to step f) of claim 13. 

On page 26, first full paragraph, with regard to claim 30, Applicant argues that 
images of products do not have to be captured using some device. The Examiner 
agrees. As Applicant has stated, the images of Fisher can be created by texture 
mapping, etc. The Examiner adds, however, that such images can also be captured. 
Further, additional reasons for combining Fisher and Reinhardt were given in the 
previous office action. 

On page 27, first full paragraph, with regard to claim 32, Applicant argues that 
the Reinhardt's product is the object in the scene, while the product being referred to in 
the claim is an object that is not present in the scene but needs to be transformed by 
the camera and integrated into the scene. The Examiner responds by first pointing out 
that there does not appear to be any recitation in the claim that the camera transform 
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the product. Furthermore, in Reinhardt's abstract, lines 1-2, the invention constructs a 
model on top of the image. This would at least imply that the product is not present in 
the scene initially. 

On page 28, first full paragraph, with regard to claim 37, Applicant argues that 
the camera transforms the image of a real world object initially outside the scene. The 
Examiner responds by pointing out that in claim 28 (from which claim 37 depends), 
does not require the camera transforming the image of the object. Further, the 
transforming act is performed on either the scene digital image or the product 
representation. Therefore, there is no requirement that the product be transformed. 
Nevertheless, Reinhardt does teach transforming the product, as discussed in the 
previous office action. 

On page 29, second full paragraph, with regard to claim 41 , Applicant argues that 
the present invention differs from Reinhardt because in the present invention, the 
product digital image used for calculating the product image capture parameters refers 
to an image of a product not in the scene and to be inserted into the scene. The 
Examiner notes, however, that claim 28 (from which claim 41 depends) does not require 
calculating product image capture parameters from the product digital image. The claim 
requires calculating the capture parameters from the scene digital image of the space. 

On page 30, last sentence, with regard to claim 42, Applicant argues that the 
present invention differs from Reinhardt because in the present invention, the camera 
information of step d) is for a product that is not present in the scene that will be 
subsequently inserted into the scene. The Examiner agrees, in light of the language of 
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the claim, including the language added by amendment. The Examiner therefore 
withdraws the rejection of claims 42-48. Claim 42 appears to differ from the other 
claims (e.g. claim 13, etc.) in that the camera information relates to the device which is 
used to capture in image of the product, whereas in the other claims, the camera 
information relates to the camera used to capture an image of the space. 

Claim Rejections - 35 USC §112 

4. Claims 57-61 are rejected under 35 U.S.C. 112, first paragraph, as failing to 
comply with the written description requirement. The claim(s) contains subject matter 
which was not described in the specification in such a way as to reasonably convey to 
one skilled in the relevant art that the inventor(s), at the time the application was filed, 
had possession of the claimed invention. 

Claim 57 recites (added language in the last clause), "the acquired camera" 
which is "acquired by calculating from the image..." There does not appear to be 
support in the original disclosure for "acquiring a camera." 

Claim 57 recites (added language in the last clause), "a camera in the space 
from which the image was captured." There does not appear to be support in the 
original disclosure for the camera being "in the space" of which it is capturing an image. 
In other words, there does not appear to be support for the camera to both capture an 
image of the space, and to be in the space. 
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5. Claims 57-61 are rejected under 35 U.S.C. 112, first paragraph, as failing to 
comply with the enablement requirement. The claim(s) contains subject matter which 
was not described in the specification in such a way as to enable one skilled in the art to 
which it pertains, or with which it is most nearly connected, to make and/or use the 
invention. 

Claim 57 recites, "the camera was acquired by calculating from the image..." The 
original disclosure does not show how a camera can be acquired by calculations from 
an image. 

Claim 57 recites (added language in the last clause), "a camera in the space 
from which the image was captured." The original disclosure does not explain how the 
camera can be "in the space" of which it is capturing an image. In other words, there is 
no explanation of how the camera can both capture an image of the space, and be in 
the space itself. This appears to be an impossibility. 

6. Claims 57-61 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. 

In the language added to the last clause of claim 57, "the acquired camera" lacks 
proper antecedent basis. The claim previously recites that an object representation is 
acquired, but does not recite that a camera is acquired. 
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Claim Rejections - 35 USC § 103 



7. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

8. Claims 13-26, 28-41 and 51-56 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the combination of U.S. Patent 6,331 ,858 to Fisher and U.S. Patent 
6,333,749 to Reinhardt et al. (hereinafter "Reinhardt"). 

As to claim 13, Fisher discloses a method for generating remote presentations of 
products in situ for a user comprising the steps of: 

a) prompting the user to provide a representation of a space in which a product is 
to be viewed in context (column 2, lines 47-50; column 5, lines 45-46); 

b) accepting a product representation representing a product for which a three- 
dimensional geometric model exists or can be created (column 5, line 43); 

e) accepting an input of a product location, the product location being a location 
in the space where the product is to be displayed in situ (column 5, lines 48-49); 

g) forming an image of the space where the modified image has been modified to 
show the identified product in situ in the image with a proper proportion and perspective 
(column 1, lines 40-41; column 2, lines 11-13; column 5, lines 48-49). 

Fisher does not disclose acquiring a digital image of the space, or the steps of c), 
d), f) and g) (as claimed). However, in an analogous environment, Reinhardt discloses: 
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acquiring a digital image of a space (column 4, lines 1-2; column 5, lines 5-8); 

c) determining at least one dimension reference, wherein a dimension reference 
is a measurement of a distance in the space corresponding to a separation of two points 
on the digital image (column 1 1 , lines 19-25); 

d) calculating, from the image, the at least one dimension reference and 
reference information, camera parameters for a camera in the space from which the 
image was captured, wherein the camera parameters include at least a camera position 
and a focal length (abstract, lines 5-6); 

f) transforming the geometric model of a product based on at least the camera 
position and the product location to form a transformed geometric model (column 3, line 
67 to column 4, line 4; column 7, lines 62-65); and 

g) combining the transformed geometric model and the digital image to form a 
modified image of the space (column 5, lines 25-27). 

Reinhardt's technique would provide more realistic images of the space 
containing the products in situ, therefore it would have been obvious to one of ordinary 
skill in the art to modify Fisher's invention according to Reinhardt. 

With regard to claim 14, Fisher discloses the method of claim 13, wherein the 
step of accepting a product representation comprises the steps of: 

prompting the user to identify a product of interest from a set of products (column 
2, lines 60-61; column 5, lines 46-50; allowing the user to select implies prompting of 
the user); 
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searching a collection of product representations to locate a member of the 
collection that matches the product identified by the user (column 2, lines 60-61 ; 
selection of one of a plurality of products implies that the a collection of such products is 
searched and located, and matches that identified by the user); and 

accepting the matching product representation (column 2, lines 63-64; column 5, 
lines 48-51). 

Regarding claim 15, repeating steps b), e) and f) for second and subsequent 
selected products would have been an obvious extension to the combined Fisher- 
Reinhardt method. Note that to only perform the steps for one product would defeat the 
purpose of the invention, and would make the invention essentially useless for retail 
transactions over the Internet. 

As to claim16, Fisher teaches prompting the user to acquire a three-dimensional 
product of interest to the user (e.g., column 5, lines 65-66; column 6, line 7). In 
combining Fisher and Reinhardt as discussed above with regard to claim 1 , a three- 
dimensional geometric model would be acquired (e.g., see Reinhardt, column 4, line 1). 

Regarding claim 17, repeating step a) for second and subsequent images of the 
space and using each of the provided images would have been an obvious extension to 
the combined Fisher-Reinhardt method. Keeping in mind that Fisher allows different 
viewpoints of the space (column 2, lines 11-13), to only perform the step for one product 
would defeat the purpose of the invention, and would make the invention essentially 
useless for retail transactions over the Internet. 
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As to claim 18, Fisher, as modified by Reinhardt discloses the method of claim 
13, wherein the modified image of the space is a two-dimensional view of a three- 
dimensional geometric model (note Fisher's column 1 , lines 44-45; and the 3D 
geometric model taught by Reinhardt), the method further comprising a step of moving 
a camera position of the modified image of the space to simulate moving around in the 
space (e.g., Fisher, column 2, lines 11-13). 

With regard to claim 19, Fisher discloses the method of claim 13, wherein the 
step of prompting the user to identify the product from a set of products is performed 
using a commerce server that serves product models and further comprising a step of 
transmitting the digital image to the commerce server (column 3, line 27). 

As to claim 20, Fisher discloses the method of claim 13, wherein the step of 
prompting the user to identify the product from a set of products is performed using a 
commerce application that receives product models and further comprising a step of 
providing the digital image to the commerce application (column 5, line 66 to column 6, 
line 2). 

With regard to claim 21 , Reinhardt further teaches that the camera parameters 
include camera position, camera rotation (orientation), focal length (abstract, lines 5-6). 
Reinhardt does not explicitly mention that a camera parameter includes center of 
projection. However, the Examiner takes Official Notice that the camera parameter 
center of projection is well known. It would have been obvious to calculate this camera 
parameter because it would provide an enhanced 3D scene. 
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As to claim 22, Reinhardt further teaches that the reference information includes 
correspondences between two-dimensional image features and three-dimensional 
structures (column 11, lines 19-25; the points are in the 2D image, distance is taken in 



As to claim 23, Reinhardt further teaches that the step of determining the at least 
one dimension reference is a step of inputting the at least one dimension reference, 
wherein a dimension reference is a measurement of a distance in the space 
corresponding to a separation of two points on the digital image (column 1 1 , lines 19- 



As to claim 24, Reinhard further teaches that the step of determining the at least 
one dimension reference is a step of assuming a default scale and using the default 
scale to determine the at least one dimension reference column 1 1 , lines 19-25, the 
default scale is implied by the ruler distance). 

With regard claim 25, Fisher discloses the method of claim 13, wherein the three- 
dimensional model for a product is a planar representation of an object and a texture 
map to be applied to a surface of the object (column 4, lines 39-40). 

With regard to claim 26, reference is made to the discussion provided above for 
claims 13 and 19. Specific structure, such as storage, engines, etc., are considered 
inherent in Fisher and Reinhardt, given that they are computer based systems, and any 
computer programmed to perform the indicated method would provide the required 
structure. 



3D). 



25). 
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Claim 28 is similar to claim 13. The discussion provided above for claim 13 is 
applicable to claim 28. 

With regard to claim 29, Fisher discloses the method of claim 28, wherein the 
product representation includes a texture map associated with the product (column 4, 
lines 13-1 4, 39-40). 

As to claim 30, Fisher is silent with regard to the product representation includes 
at least a product digital image, captured by a product image capture device positioned 
to capture a digital image of the product. However, the Examiner contends that the 
images of the products must be captured using some kind of device. Further, Reinhardt 
teaches utilizing a digital camera to capture a digital image of objects (column 7, lines 
18-19). It would have been obvious to utilize a digital camera as taught by Reinhardt in 
Fisher's invention because of the convenience provided by digital cameras. 

As to claim 31 , Reinhardt further teaches that the product representation includes 
a capture position, capture angle of rotation and focal length (abstract, lines 5-7; note 
that these parameters are derived from the images, implying that the representation 
includes them). 

With regard to claim 32, Reinhardt further teaches that the product 
representation includes product image capture parameters including at least a 
representation of the position of the product image capture device when the digital 
image of the product was captured (abstract, lines 5-7; note that these parameters are 
derived from the images, implying that the representation includes them). 
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With regard to claim 33, Reinhardt further teaches that the product image capture 
parameters include projections of geometric elements from the product onto the product 
digital image (column 5, lines 25-28). 

With regard to claim 34, Reinhardt further teaches that the capture parameters 
for the image of the scene include projections of geometric elements from the scene 
onto the scene digital (column 5, lines 25-39). 

As to claim 35, Reinhardt further teaches that the capture parameters for the 
image of the scene include an angle of rotation for the image capture device in the 
space (abstract, lines 5-7). 

As to claim 36, Fisher, as modified by Reinhardt, discloses the method of claim 
28, wherein the step of inputting a product representation is preceded by the steps of: 
capturing a product digital image using the product image capture device (note as 
above, Reinhardt provides for the product image capture device); and 
calculating, from the product digital image, product image capture parameters that 
include at least a representation of the product image capture position and a 
representation of the product image capture focal length (note as above, Reinhardt 
teaches this, e.g., in the abstract, lines 5-7). 

As to claim 37, Reinhardt further teaches that the step of transforming results in 
the one or more transformed digital images sharing common capture parameters, the 
common capture parameters including at least a capture position (column 1 1 , lines 1-7; 
column 12, lines 12-14). 
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As to claim 38, Fisher discloses the method of claim 28, further comprising a step 
of prompting the user to identify the product from a set of products for which a three- 
dimensional geometric model exists or can be created (column 2, lines 60-61; column 5, 
lines 46-50; allowing the user to select implies prompting of the user). 

As to claim 39, see the discussion above for claim 23. 

As to claim 40, repeating step of combining would have been an obvious 
extension to the combined Fisher-Reinhardt method. To only perform the step for one 
product would defeat the purpose of the invention, and would make the invention 
essentially useless for retail transactions over the Internet. 

With regard to claim 41 , see the remarks above for step d) of claim 13. 

With regard to claim 51 , reference is made to the remarks provide above for 
claims 13, 19 and 26. 

As to claim 52, remarks analogous to those provided above for claim 14. 

Regarding claims 53-56, having the various components in either the server or 
the client is seen as a decision based upon designer preference. A designer would 
have a particular component in either the client or the server based on his or her needs 
for the given application. This is not considered a patentable difference from the prior 
art. 



Allowable Subject Matter 

9. Claims 42-48 allowed. 
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Conclusion 



Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jon Chang whose telephone number is (703)305-8439. 
The examiner can normally be reached on M-F 8:00 a.m.-6:00 p.m.. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on (703)308-6604. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 



Contact Information 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 




<fon Chang 
Primary Examiner 
Art Unit 2623 



Jon Chang 
June 22, 2004 



